Semantic Overlay Networks for Peer-to-peer Web Search

We consider a network of peers, where each peer has its own collection obtained by individually crawling the web. When designing a distributed search system for such networks, an important task is how to efficiently perform query routing, i.e., how to find the most promising peers to answer the query. However, the efficiency of those routing techniques depends heavily on the underlying network organization. Therefore, previous works have proposed the creation of semantic overlay networks (SON), where peers are grouped according to their contents. In this work we present a rather different notion of SONs, where peers are free to decide to which peers they want and to which peers they do not want to establish a connection, following the idea of the p2pDating algorithm that was recently proposed. We consider a P2P network that uses Pastry as the underlying network infrastructure, and where peers use Nutch to perform web crawls and Lucene to build local indexes. We show that SONs can greatly reduce the amount of traffic needed for answering a query, while still maintaining a high recall.